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Tone scale processing! based on image modulation activity 



(57) A method for generating an image-dependent 
tone scale curve includes (a) computing an image ac- 
tivity histogram for an input image value as a. function of 
the number of neighboring pixels that have a predeter- 
mined amount of image modulation exclusive of a pre- 
determined noise modulation; and (b) constructing an 
image-dependent tone scale curve from the image ac- 
tivity histogram. The predetermined amount of image 
modulation may comprise any image value within two 
specified intervals each bounded by a noise threshold 



and a modulation threshold. In another aspect, atone 
scale curve that optimizes the rendering of image activ- 
ity' is generated under one or more constraints. More 
particularly, one or more slope-limit constraints and/or 
point constraints are established and then atone scale 
function is constructed by applying the slope-limit con- 
straints in atone scale derivative space. An image ac- 
tivity histogram is computed and applied to the tone 
scale slopefunction. Finally, the tone scale slope func- 
tion is integrated to generate the tone scale curve. 



CM 
< 

in 
o 

CO 




COMPUTE 
IMAGE VALUE 
HISTOGRAM 



-IO 



compute: 

INPUT OtNAMlC 
RANGE 



COMPUTE 
COLOff 8 DENSITY 
BALANCE 



-14 



16 



COMPUTE 
WHITE -PIXEL 
HISTOGRAM 



19 



COMPUTE 
SKIN-PtXEL 
HISTOGRAM 



20- 



FIG. I 



COMPUTE 
IMAGE ACT/V/TY 
HISTOGRAM 



22 



OUTPUT MEDIUM 
DYNAMIC RANGE 





24— 




COMPUTE , 




CONSTRUCT 
TONE SCALE 


' ANCHOR POINT 






CURVE 











28 



SLOPE 8 POINT 
CONSTRAINTS 



TONE SCALE CURVE 
IL00WP TABLE) 



CL 
Ill 



Printed by Jouve, 75001 PARIS (FR) 



1 



EP 1 130 545 A2 



2 



Description 

[0001] This invention relates, to the field of image 
processing, and in particular to tone scale processing of 
digital images. 

[0002] The dynamic range of a scene or an image is 
the ratio of its maximum intensity to its minimum inten- 
sity. Similarly, the dynamic range of a visual display, 
such as a color monitor, or a print medium, such as pho- 
tographic paper, is the ratio of the maximum luminance 
to the minimum luminance that is rendered on the dis- 
play orthe print medium. In conventional optical printing, 
when an image has a larger dynamic v range than can be 
rendered on an output medium, there will.be loss of de-: . 
tails in the rendering process, especially in the highlight . 
or shadow areas. Digital image processing can be used 
to preserve the image details by compressing the input 
dynamic range only on. the very low frequency compo- , 
nent in the inputimage. U.S: Patent 5,01 2,333 teaches 
a method for such an operation. An < input image is de^ 
composed into a high frequency component. and a low 
frequency component. The image value Jn the low fre- 
quency component is- mapped* through a lookup table ( 
which compresses the dynamic range by a tone scale 
curve with an average slope. less than one. A tone scale 
curve, as the term is used herein, is a function that maps 
an input image value to an output image t value. When, 
the units used for the input and the output image values 
are both proportional to log exposure or lightness, of the. 
scene, the slope of the tone scale curve is known to. cor^ 
relate well with the perceived contrast of the image.,, 
[0003] ' It is desirable to have an automatic method of 
constructing an image-dependent tone scale curve for 
a given input digital image such that the dynamic range 
of the input image is so compressed as to reader, most 
image details visible. For example, if the image contains 
a highlight area that is illuminated by bright sunlight and 
a shadow area that is illuminated only by dim skylight, 
it would be desirable that detail in both areas would be 
rendered visible. Because of the large difference in the 
luminance level of the two illuminations, the tonal range 
of the highlight is weH separated from that of the shadow. 
If the histogram of the input image values is examined, 
onecari see a gap between the highlightandthe shad- 
ow clusters. In this case, the tonal range corresponding 
to the gap can be greatly compressed without creating 
undesirable artifacts because there are no pixels within 
that tonal range. The limited" output dynamic range 
should be fully allocated to the highlight tonal range and 
to the shadow tonal range, with nothing wasted.for the 
tonal range corresponding to the gap in between. 
[0004] Prior algorithms for dynamic range allocation, 
such as the well known method of histogram equaliza- 
tion, are based on the histogram of pixel values of the 
entire image. There are two drawbacks with this type of 
approach: (a) a large uniform area (such as sky or a wall) 
in the image will cause a large output dynamic range to 
be allocated for that area. This is not desirable because 



the area is uniform and does not need much output dy- 
namic range; (b) the pixels corresponding to a large 
peak in the histogram may not be spatially adjacent to 
each other in the image. Although collectively, their 
5 number may be very large, the image values of their 
neighboring pixels may be quite different from them and 
therefore they do not need a large output range for ren- 
dering. 

[0005] To solve the first of these problems, U.S: Pat- 

10 ent number 4,654,722 teaches a method of using a La- 
placian operator to select only edge pixels for inclusion 
in the histogram. Therefore, most pixels in the large uni- 
form areas are not included. Such a method can reduce 
the problem of large uniform areas, but does not truly 

15 measure image signal modulations. U.S. Patent 
Number 5,633,511 teaches a method that is based on 
the histogram of image activities, not on image values. 
Image activity measures how much signal modulation 
takes place at each image value. Although the '51 1 pat- 

20 ent teaches the use of image activity to set the param- 
eter values in a family of tone scale curves, it does not 
teach the use of image activity to allocate the output dy- 
namic range to the tone ranges in the input image values 
so that ah image-dependent tone scale curve of a non- 

25 predetermined shape can be constructed. 

[0006] The present invention is directed to overcom- 
ing one or more of the problems set forth above. Briefly 
summarized, according to one aspect ofthe present in- 
vention, a method, of generating an image-dependent 

30 tone scale curve includes (a) computing an image ac- 
tivity histogram for an input image value as a function of 
the number of neighboring pixels that have a predeter- 
mined amount of image modulation exclusive of a pre- 
determined noise modulation; and (b) constructing an 

35 image-dependent tone scale curve from the image ac- 
tivity histogram. The predetermined amount of image 
modulation may comprise any image value within two 
specified intervals each bounded by a noise threshold 
and a modulation threshold. 

40 [0007] In another aspect of the present invention; the 
invention involves a method of generating a tone scale 
curve that optimizes the rendering of image activity un- 
der one or more constraints. More particularly, the in- 
vention involves establishing one or more slope-limit 

45 constraints and then constructing a tone scale slope 
function by applying the slope-limit constraints in a tone 
scale derivative space. An image activity histogram is 
computed and applied to the tone scale slope function 
Finally, the tone scale slope function is integrated to 

so generate the tone scale curve. In yet another aspect, 
the invention involves establishing one or more point 
constraint^ and then constructing a tone scale slope 
function in a piecewise manner by applying the point 
constraints, in the tone scale derivative space. The im- 

55 age activity , histogram is computed and applied to the 
tone scale slope function, and the tone scale slope func- 
tion is integrated to generate the tone scale curve. 
,[0008] In a further aspect, the output dynamic range 
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(Vmin, Umax) of an output medium is measured. For an 
input digital image, two histograms are computed: the 
image activity histogram and an image value histogram. 
The minimum image value Umin and the maximum val- 
ue Umax of the input image are determined from its im- 
age value histogram. A balance point is also determined 
for the image, such that a certain image value Uaim is 
tentatively mapped to itself with an output value Vaim = 
Uaim. A white point, Uwhite, is also determined by de- 
tecting the presence of any white object in the image. 
The input as well as the output dynamic range are both 
partitioned into the upper range and the lower range by " 
the mapping of a chosen anchor point. 
[0009] These and other aspects, objects, features 
and advantages of the present invention will be more 
clearty understood and appreciated from a review of the 
following detailed description of. the preferred embodi- 
ments and appended claims, and by reference to the 
accompanying drawings.. 

[0010] FIG. 1 is a diagram of a method for tone scale 
construction based on image modulation activity ac- 
cording to the, invention. 

[0011] FIG. ,2 is a block diagram showing the function 
of a tone scale curve for mapping an input image value 
to an output image value. 

[0012] FIG, 3 is an example ; of the characteristics of 
an output device, showing how the device input values, 
specified by an output dynamic Vange represented by 
the deyice input, values Vmin and Vmax, are printed 
close to. the extreme densities Dmin and Dmax on the 
output medium. - , 

[0013] FIG. 4 shows an example of how a tone scale 
curve is constructed from an image activity histogram. 
[0014] FIG. 5 t is a schematic diagram of conditional 
probability distributions of white points and sKin points 
in a luminance-chrominance color space. 
[00151 FIG. 6 shows an example of a preferred meas- 
ure of image activity used for the image activity histo- 
gram shown in Figure 4. ....... . - 

[001 6] FIG. 7 is a diagram of a preferred tone scale 
curve construction process according to the invention. 
[001 7] FIG. 8 is a diagram of a computer system for 
implementing the present invention. 
[001 8] • Because image processing systems employ- 
ing tone scale correction are well known, the present 
description will, be directed in particular to attributes 
forming part of, or cooperating more directly with, appa- 
ratus in accordance with the present invention. System 
attributes not specifically shown or described herein 
may be selected from those known in the art. In the fol- 
lowing description, a preferred embodiment of the 
present invention would ordinarily be implerhented as a 
software program, although those skilled in the art will 
readily recognize that the equivalent of such software 
. may also be constructed in hardware. Given the'system 
and methodology asdescribed in the following materi- 
als, all such software implementation needed for prac- 
tice of the invention is conventional and within the ordi- 



nary skill in such arts. If the invention is implemented as 
a computer program, the program may be stored in con- 
ventional computer readable storage medium, which 
may comprise, for example; magnetic storage media 
5 such as a magnetic disk (such as a floppy disk) or mag- 
netic tape; optical storage media such as an optical disc, 
optical tape, or machine readable bar code; solid state 
electronic storage devices such as random access 
memory (RAM), or read only memory (ROM) ; or any oth- 
w er physical device or medium employed to store a com- 
puter program. 

[0019] This invention relates to a method of construct- 
ing a tone scale curve for rendering digital images on a 
display or a print medium that has a limited output dy- 
15 namic range. The invention can be used in various ap- 
plications where digital images are processed and dis- 
played or printed,^ such as in digital printing, photofinish- 
ing, medical imaging, video imaging, .motion pictures, 
and graphic arts. This invention applies to all image dis- 
20 play devices (such as cathode-ray tube monitors or liq- 
uid-crystal displays) orprint media (such as photograph- 
ic paper or inkjet paper). In order to simplify the.descrip- 
tibn, photographic paper will be used as the output me- 
dium in the following discussion. 1 
25 [0020] Figure i is a block diagram showing the overall 
methodology of the invention. In an image value histo- 
gram generation stage 10, an input image is used to 
compute an image value histogram, from which a com- 
putation stage 12 determines. the input dynamic range. 
30 in a scene balance stage 1 4, a scene balance algorithm 
is used to compute the color and density balance for the 
input ; image. Such scene balance algorithms are well- 
known and described in U.S. Patent Nos. 4,945,406 and 
5,016,043 and in C. J. Bartleson and R. W. Huboi, "Ex- 
35 po'sure Determination Methods for Color Printing: The 
Concept of Optimum Correction Level", J. SMPTE, 65: 
205-215 (1956), which are incorporated herein by refer- 
ence. The output of the scene balance algorithm in- 
cludes color balance points for the. red, green, blue 
40 records, from which the algorithm computes the lumi- 
nance balance point (aim point) and the chrominance 
balance point. These balance points are used in the his- 
togram computation stages 16 and 18 to compute a 
white-ptxePhistogram and a .skin-pixel histogram, ,re- 
45 spectlvely. In'the image activity stage 22, an image ac- 
tivity histogram is computed from the input image. In one 
of the main aspects of the invention, a tone scale con- 
struction stage 24 constructs a tone scale curve, using 
the information computed from the other stages and al- 
so so parameters 26 about the dynamic range of the output 
medium input and* as will also be described later, sev- 
eral slope and point constraints 28. In the preferred em- 
bodiment of this invention, a balance point derived from 
th'e color and density balance algorithm 1 4 and the white 
55 point requirements derived from the white-pixel histo- 
gram 16 are merged into one anchor point constraint 
which is solved iteratively in an anchor point stage 20 
and'used in the tone scale construction, as will also be 
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described later. . - . ■ 

[0021] Figure 2 shows the function of a tone scale 
curve 30 f which maps every input image value u to an 
output image value v, which is used by the output device 
to determine the density of the reflection print. Then, as 
shown in an output curve 32, the output device reads 
the output image value v and produces a density D on 
the reflection print of the image. If the input image is cal- 
ibrated in log exposure of the scene, then u has the unit 
of log exposure. If the output device, such as a laser 
writer, is also calibrated to receive log exposure as the 
input metric, then v also has the unit of log exposure. 
The tone scale curve is thus a mapping from log expo- 
sure to log exposure. If the input image value u is in print- 
ing density of a color negative film, and the output device 
is also calibrated to receive printing density value v, then 
the tone scale curve is a mapping from printing density 
to printing density. Therefore, for purposes of this inven- 
tion, the inpul and the output variables of the tone scale 
curve have the same physical units. The same tone 
scale curve can be used to drive different output devices 
if they are calibrated to receive the metricfthat the input 
image uses. The choice of the metric to use is flexible 
as long as the slope of the tone scale curve is correlated 
with the perceived image contrast. For example, relative 
log scene exposure, printing density, and relative scene 
lightness (such as li* in CI ELAB space), are all valid 
metrics for purposes of this invention. 
[0022] The output dynamic range is determined for 
the output medium in terms of its maximum and mini- 
mum device input values v that will be printed to certain 
chosen densities. Figure 3 shows an example of an out- ; 
put curve with specific output device/medium character- 
istics. The minimum density Dmin and the maximum 
density Dmax are extreme density values that the output 
device can produce. The two device input values Vmin 
and Vmax are mapped to density values close to, but 
may or may not be equal to, the extreme densities, der 
pending upon the margins that are reserved to produce 
highlight and shadow modulations;. In thjs example, -, 
Vmin and Vmax specify the available output dynamic 
range for the tone scale curve 30 (Figure 2). The input 
dynamic range is determined from the input image by 
the input image histogram stage 1 0 and dynamic range 
computation stage 12 (Figure 1). The histogram of the. > 
input image values is accumulated in the stage 10 and, 
in practice, the 0.2 and the 99.8 percentile values, re- - 
f erred to as Umin and Umax, are taken in the stage 1 2 
to define the input dynamic range. Other percentile val- 
ues, of course, may be used to define the input dynamic 
range. ' 

[0023] The basic idea of the invention is described 
with reference to the tone scale curve and image activity 
histogram shown in Figure 4. The input dynamic range 
is shown to be from Umin io Umax, and the output dy- 
namic range is shown to be from Vmin to Vmax. The 
simplest tone scale method to map the input to the out- 
put is through the dashed straight line C1 , which allo- 



cates the available output dynamic range Vmax-Vmin 
uniformly to the input dynamic range Umax-Umin, re- 
gardless of the content of the image. One of the key fea- 
tures of this invention is that the image activity should 
5 be computed for each input image value and its histo- 
gram should be used to guide the allocation of the avail- 
able output dynamic range to the input dynamic range. 
More particularly, as will be described, the image activity 
should be based on a specified degree of image modu- 

10 lation in the neighborhood of each pixel. Therefore, 
whenever the image activity is low for a range of input 
image values, the slope of the tone scale curve should 
be made small. Whenever, the image activity is high, the 
slope should be increased proportional to the height of 

15 the image activity histogram. In Figure 4, the bottom 
curve C3 shows the image activity histogram of an input 
image. The top curve C2 shows a tone scale curve that 
has a high local slope at regions A and C for image val- 
ues where there are a lot of activities in the input histo- 

20 gram and a low slope at region B for image values where 
there are few activities in the input histogram'. 
[0024] . The. image activity . can be calculated in a 
number of ways. For example, the response at a pixel 
of the input image to a bandpass or a highpass filter can 

25 be used as a measure of image activity for the image 
value at that pixel. In U.S. Patent Number 5,633,511 , 
which is incorporated herein by reference, image activity 
is measured by determining activity values based on 
level-crossing, image value co-occurrence, and the use 

30 of edge gradient and Laplacian operators; While these 
techniques work well, with edges, they do not adequately 
measure. texture - such as would be found in hair, grass 
and fabric, micropattems. In order to measure both edg- 
es and textures, an image modulation activity measure 

35 is used according to the invention to find, in an N x N 
neighborhood, the number of pixels in the neighborhood 
of the, center pixel that, have a predetermined image 
modulation exclusive of noise modulation. Pixels that 
are spatially adjacent and have similar image values can 

40 be rendered more distinguishable if the local slope of 
the tqne scale curve is made high at the image value. 
In a uniform area, the image values in the neighborhood 
will fluctuate, because of noise. Therefore, pixels that 
have image values that are within two or three noise 

45 standard deviations from the center pixel should not be 
counted. Likewise, pixels that have image values' far 
away from the center pixel also should not be counted 
because they are already quite different from the center 
pixel - i.e., there is no need to adjust the local slope to 

50 make their differences more visible. Therefore, in com- 
puting the image activity,, two boundaries are set: a lower 
bound (determined by noise) and an upper bound (de- 
termined by the system minimum contrast requirement). 
The activity value. for each central pixel image value is 

55 then comprised of the number of bounded pixels, i.e., 
that are within the aforementioned upper and lower 
bounds for an N x N region around the central pixel; and 
tha histogram is then jncremented for each input image 
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value by the number of bounded pixels for the given im- 
age value of the central pixel. 

[0025] Thus, in the preferred embodiment of the in- 
vention, the image activity is measured by counting, for • 
each pixel in the input image, the number of neighboring s 
pixels that have an image value within the two afore- 
mentioned specified intervals, which are bounded by 
noise thresholds and modulation thresholds. Let the irrV 
age value be x, the noise threshold be n, and the mod- ' 
ulation threshold be m. Then the two intervals are spec- 1 o 
ified by [x-m, x-nj and /x+n, x+mj. The noise threshold 
can be determined' for each input type, e.g., by measur- 
ing the granularity of various films or by a noise estima- 
tion algorithm that operates on the input image. Typical 
noise estimation algorithms are described in U.S. Patent « 
No. 5,923,775 and in S. I. dlsen, "Estimation of Noise 
in Images: An Evaluation", CVGIP: Graphic Models and 
Image Processing, 55, 4, 319-323, 1993. The modula- 
tion threshold is not a sensitive parameter. For an image 
calibrated in 1000 * log exposure, and represented by so 
integers between 0 and 4095, a value of m= 200 is found - 
to give good performance. FIG. 6 shows an example of 
how the image activity is calculated for a 5x5 window < 
withVi = 40 and m= 200! There are 1 0 neighboring pixels 
in the window that have values within the two specified '25 
intervals [x-200 t x-40]an6 [x+40, x+200], where x= 593 
is the image value of the center pixel. Therefore, the im- 
age activity is 1 0, and the histogram is incremented by 
10 for the image value 593. \ ' ' 

[0026] Furthermore, in order to produce pleasing tone & 
reproduction it is desirable to impose several additional * 
constraints on this basic idea in the construction of the 
tone scale curve according to the image activity histo- 
gram. These constraints are discussed below: ' 35 

• The first constraint is that certain input Values have 1 
to be mapped to certain output values so that the' 
"'lightness of the reproduced image will be; properly 
rendered. This is called the density balance prob- 
lem. For example, the faces in the image have to 40 
be printed within a proper lightness range. Other- 
wise, the image will appear to be too dark or too 
light. As shown in Figure 2, a scene balance algo- 
rithm stage 14 cain be integrated witrvthis invention 
in order to compute the color and density balance « 
points. Alternatively, a balance (aim) point Lte/mcan 
be determined from the image by a separate scene 
balance algorithm and the balance (aim) point Uaim 
' would then be an input parameter to the tone scale 
construction process according to this invention, so 
Typically, the balance (aim) point is set to be at or 
' near the 18% gray point. 
. ' s The second constraint is that main subjects in the 
image have to be* rendered with proper contrast. 
Most of the* time, the main subjects in the image are* 55 
people. In this invention, the input tonal range that' 
contains potential skin-color pixels rs detected (by 
the skin-pixel histogram stage 18 shown in Figure 



1 and described below) and the tone scale slope 
within that range is constrained to have user spec- 
ified minimum and maximum slopes. 
The third constraint is that a white object in the im- 
age has to be rendered with a sufficiently high re- 
flectance factor in the reflection print. Otherwise, 
the white object will appear to be grayish. Consum- 
er studies show that the sensitivity to this require- 
ment varies significantly from person to person. In 
the preferred embodiment of the invention, a white 
object detector (the white-pixel histogram stage 16 
shown in Figure 1 and described below) is used to 
detect the presence of white objects in the image. 
Typically, the white point, is set to be .about two 
stops, or 0.6 log exposure, above the balance(aim) 
point. If any white object is detected,- its correspond- 
ing input image value (white point) is constrained to 
be mapped to some high output image value. In 
practice, the input white point is mapped to itself so 
that its lightness will not be reduced by the tone 
scale curve, 

► A fourth constraint is that the highlight contrast and 
the shadow contrast can have some minimum slope 
values different from the user specified mid-tone 
minimum contrast. This is necessary so that the out- 
put look can be controlled by the user. For example, 
it is often desirable to raise the shadow contrast be :: 
cause its reproduction is affected. most. seriously by 
the viewing flare. \ 

[0027] ' The detection of skin-pixels and white-pixels is 
accomplished by compiling the conditionalprobability 
distributions of the desired color pixels from a large im- 
age database. For each pixel in the input image, itscon : . 
ditidnal probability of being skin-color or white-color is 
used to classify the pixel into either a skin-pixel histo- 
gram or a white-pixel histogram (this same technique 
involving conditional probability can also.be used to 
classify other pixels into, other histograms appropriate 
for other colors). The skin-pixel histogram is used to 
compute the tonal range of skin and the white-pixel his- 
togram is used to compute the tonal. range of white ob- 
jects. These two tonal ranges are used as the skin con- 
trast constraint and the white point constraint in, the tone , 
scale curve construction. FIG. 5 is a schematic diagram, 
of the conditional white distribution 40 and skin distribu- 
tion 42 in (L,s,t) color space, where L = c(R+G+B); s = 
a(R-b); t = b(R-2G+B); a, b and c are constants; and R, 
G and B are image values proportional to relative log 
scene exposure. The actual shapes of the conditional 
distribution are determined by the compiled statistics. 
They are <not: necessarily ellipsoid or rectangular. 
[0028] The first constraint of the balance point and the 
third constraint of the white point can be imposed simul- 
taneously: Then the tone scale curve has to pass both 
points. This means that the curve is constructed piece- 
wise in three pieces: one piece below the balance point, 
one piece above the white point, and one piece between 
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the two points. For many images, the three-piece con- 
struction leaves little room to compress the highlight and 
the shadow tonal ranges. In the preferred embodiment 
of this invention, the balance point and the white point 
requirements are merged into one constraint which is 
solved iteratively. An anchor point is initially set at the 
white point, the curve is constructed piecewise as two 
pieces, one above the anchor point and one below. The 
balance point is then mapped through the constructed 
curve. If it is mapped too far away from its aim point, 
then the anchor point is adjusted and a new curve is 
constructed through the new anchor point. This process 
is iterated several times until the anchor point has been 
adjusted so- that the balance point is mapped within the 
specified distance of its aim point. ^ 
[0029] The above description shows that there are 
many constraints to be satisfied in the construction of 
the tone scale curve based on the image activity histo- 
gram. Therefore another key feature of this invention is 
a method of constructing a curve that must 'Simultane- 
ously satisfy several slope and point constraints. This is 
difficult to accomplish in tone scale space, but easier to 
satisfy in atone scale derivative space, that is, with a 
tone scale slope function. Therefore, in the preferred 
embodiment of the invention, the constraints are satis- 
fied in derivative space and the tone scale curve is con- 
structed by integrating its slope function. Since the slope 
function is the function that is computed first, all the 
slope constraints are easy to impose. The point con- 
straints are also easy to enforce by using a scale factor 
which is adjusted so that when the slope function is in- 
tegrated, the resulting curve passes through the re- 
quired pbint. 

[0030] FIG. 7 shows a block diagram of the steps of 
the preferred embodiment of this invention for construct- 
ing a tone scale curve based on the image activity his- 
togram and the aforementioned constraints. The image 
values are assumed to be integers between 0 and 4095; 
representing 1000 * log exposure, or 1000 * printing 
density. The first step 100 is to initialize three working 
arrays: The minslope array is used by the user and the 
computer algorithm to impose the minimum allowed 
slope for each image value. The maxslope array is for 
imposing the maximum allowed slope. The slope array 
is the first derivative of the tone scale curve to be con- 
structed. The next step 110 is to impose the minimum 
and maximum skin contrasts for the tonal range of the 
detected skin histogram'by setting the minimum slopes 
and the maximum slopes for the image values corre- 
sponding to skin in the minslope array and the maxslope 
array In the next step 120, the balance (aim) point and 
the white pixel histogram are used to initialize the anchor 
point. In order to compensate for viewing flare or some 
undesirable characteristics of the output device, the 
minimum slope in the highlight and shadow areas can 
be raised in step 130 by setting the desired values in the 
minslope array. At this stage, other kinds of constraints 
can' also be imposed in a similar manner. The algorithm 



then starts an iterative process 142 to construct the tone 
scale curve. In performing each iteration, the balance 
(aim) point as mapped by the computed tone scale 
curve is checked in step 180 to see if it is shifted too far 

5 away from its original aim. If the shift is too large i then 
the computer adjusts the anchor point, and goes into the 
next iteration, until ;a tone scale curve has been 
achieved which does not produce a shift in aim point 
larger than specified by the user. This iterative process 

10 is guaranteed to converge in N steps by adjusting the 
anchor point closer to the aim point by one-Nth the in itiaf 
distance between the anchor point and the aim point. 
[0031] It is helpful at this point to understand that the 
object of this invention is to construct an image-depend- 

15 ent tone scale curve such that: 

• The input dynamic range is always mapped by the 
curve to a dynamic range smaller or equal to the 
dynamic range of the output medium. 

20 • |f the input upper range is greater than the output 
upper range, then, a curve with average, slope less 
than. 1.0 is constructed so that the instantaneous 
v slope is afunction of the height of the image activity 
histogram, and Umax is mapped to Vmax, subject 

25 to .the user, specified slope and point constraints. 

• . ;■ If the input, upper range is less than or equal to the 

..output upper, range, then a straightline with slope 
greater than, or equal to 1 .0 is used for the upper 
part of the tone scale. curve. ' , 
30. • . r |f the input Ipwer range is greater than the output 
lower range, then a curve with average slope less 
than 1.0, is constructed so that the instantaneous 
- slope is a function of the height of the image activity 
histogram, and Umin ls mapped to Vmin, subject tb 
35 the user specified slope and point constraints. 

• If ithe. input lower range is less than or equal to the 
output lower range, then a straightline with slope 

* greater than or equal to 1 .0 is used for the lower 
, < part of the tone scale curve. The slope and point 
40 constraints are either specified by the user or geh- 
t< erated by object detection modules in the tone scale 
. construction algorithm, such as in the skin detection 
stage : 1 8 and white object detection stage 1 6 shown 
jn Figure 1. 

[0032] Referring again to Figure 7, the construction of 
the tone scale curve starts in step 140 by partitioning 
the input and the output dynamic range into the upper 
and the lower range at the anchor point, which is 

50 mapped into itself. Therefore the tone scale curve is 
constructed in two pieces, one for the upper range and 
one for the lower range, with the same procedure. Here 
we describe the procedure with respect to the upper 
range. First, the minslope array is copied over to the 

55 slope array and the integral of the minslope array in the 
upper input range is subtracted in step 150 from the 
available upper output dynamic range. If the result is ze- 
ro or negative, then the upper curve is constructed by 
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integrating the minslope array. If the result is greater 
than zero, then the difference is distributed to the upper 
input range according to the image activity histogram in 
the upper range. Image values that have higher image 
activity are allocated more output dynamic range by 
having higher values set in the slope array. The slope 
array is then checked against the maxslope array. The 
checking starts at the anchor point. If any value ,n the 
slope array is larger than the maxslope value, then the 
slope is set to the maxslope value and the difference is 
added to the next erttiy in the slope array. After this 
check-and-clip process is done, the slope array is inte- 
grated in step 170 to obtain the upper part of the tone 
scale curve for the upper range. The same procedure is - 
repeated for the lower range to construct the lower part 
of the tone scale curve. Once the entire curve is con- 
structed and the aim point is checked to be within spec- 
ification, the tone scale curve is produced as a lookup 
table. The input image is mapped through the tone scale 
lookup table to produce the desired output image. 
[00331 ' "While the overall methodology of the invention 
is described above, the invention can be embodied in 
any number of different types of systems'&nd executed 
in ahy number of different ways, as would be known by 
one ordinarily skilled in the art. For example, as illustrat- 
ed in Figure 8, a typical hardware configuration^ an 
information handling/computer system in accordance 
with the invention preferably has at least one processor 
or centra, processing unit (CPU) 200. The CPU 200 ,s 
interconnected via a system' bus 201 to a random ac- 
cess memory (RAM) 202. a read-only memory (ROM) 
203, an input/output (I/O) adapter 204 (for connecting 
peripheral devices, such as disk units 205 and tape 
drives 206 to the bus 201). a communication adapter 

207 (for connecting an information handlingsystem to 
a data processing network), a user interface adapter 

208 (for connecting peripherals 209, 210, 21V such as 
a keyboard, mouse, digital image input unit and/or other 
user interface device to the bus 201 ), a printer 21 2 and 
a display adapter 213 (for connecting the bus 201 to a 
display device 21 4).The invention could be implement- 
ed using the structure shown in Figure 8 by including 
the inventive method within a computer program stored 
on the storage device 205. Such a computer program 
would act on an image frame supplied through the ^in- 
terface adapter 208 or through the network connection 
207 The system would then automatically produce the 
desired tone scale response on the processed image, 
which would be output to the display 21 4, the pnnter 
212 the network 207 orwhatever otheroutput medium/ 

• device would be warranted under the circumstances. 



Claims 



1 ' A method of generating an image-dependenHone 
scale curve in orderto enhance a digitahnput image 
comprised of a plurality of image pixels, each pixel 



characterized by a image value that is input to the 
tone scale curve, said method comprising the steps 
of: 

(a) computing an image activity histogram for 
an input image value as a function of the 
number of neighboring pixels that have a pre- 

■ determined amount of image modulation exclu- 
sive of a predetermined noise modulation; and 

(b) . constructing an image-dependent tone 
scale curve from the image activity histogram. 

2 The method as claimed in daim 1 wherein the pre- 
determined amount of image modulation computed 

»5 in step (a) comprises an image value within two 
. . specified intervals each bounded by a noise thresh- 
old and a modulation threshold. 

3 A method of generating a tone scale curve that op- 
20 • timizes the rendering of image activity under, one or 

more constraints, said tone scale curve operating 
:uP on a digital input image comprised of a plurality 
of image values that are input to the tone scale 
. curve, said method comprising the steps of : 

25 (a) establishing one or more slope-limit con- 

straints; • . 

(b) constructing a tone scale slope function by 

■ . applying the slope-limit constraints in a tone 
30 scale derivative space; 

(c) computing an image, activity histogram f rorn 
a measure of image activity; ; s 

(d) applying the image activity histogram to the 
tone scale slope function; and 

35 . , (e) integrating the tone scale slope function ,tp 

• generate the tone scale curve. : „ 

* The.method.as claimed in claim 3 further compris- 
ing: i.-- ' = - ; ' 

| . , , (f) establishing one or more point constraints; 

)i . and : ' . ■ . •" . 

, (g) constructing the tone scale slope function in 
■a piecewise manner.by applying the point con- 
straints in the tone scale derivative space. 

5 The method as claimed in claim 4 wherein the point 
constraints comprise at least one of a balance point 

, and a white point. ,: . , .. 

6 The, method claimed in claim 3 wherein the' image 
. activity measure in step (c) is measured by the. re- 
sponse of the input image to a bandpass or high- 
pass filter.' . •■. 

7 The method claimed in claim 3-wherein the image 
activrty measure in step (c).is measured. by the gray 

. . . level co-occurrence statistics of the input image. 



40 



45 



50 



55 



7 



13 EP1 130 545 A2 

8. The method claimed in claim 3 wherein the image 
activity measure in step (c) is measured by count- 
ing, for each pixel in the input image, the number of 
neighboring pixels that have an image value within 
two specified intervals, which are bounded by noise s 
thresholds and modulation thresholds. 

9. The method claimed in claim 3 wherein the image 
activity measure in step (c) is measured by the re- 
sponse of the input image to an image gradient op- w 
erator. 

10. The method claimed in claim 3 wherein the image 
activity measure in step (c) is measured,by counting 
the number of level-crossings for each Input image is 
value. 

11. The method as cjaimed in claim 3 wherein the 
slope-limit constraint comprises a skin contrast con- 
straint. . 20 

12. A method of generating a tone scale curve.that op-„ 
timizes the rendering of image activity under one or 
more constraints, said tone scale curve operating 
upon a digital input image comprised of a plurality 25 

of image values that are input to the tone scale , \ 
curve, said method comprising the steps of: ' 

(a) establishing one or more point constraints; ; 

(b) constructing a tone scale slope function in 30 . 
a piecewise manner by applying the point con- . 
straints in a tone scale derivative space; 

(c) computing an image activity histogram from 
a measure of image activity; 

(d) applying the image activity histogram to the 35 
tone scale slope function; and 

(e) integrating the tone scale slope function to " 
generate the tone scale curve. 
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